Goto

Collaborating Authors

 equalized odds




Paradoxes in Fair Machine Learning

Neural Information Processing Systems

Equalized odds is a statistical notion of fairness in machine learning that ensures that classification algorithms do not discriminate against protected groups. We extend equalized odds to the setting of cardinality-constrained fair classification, where we have a bounded amount of a resource to distribute. This setting coincides with classic fair division problems, which allows us to apply concepts from that literature in parallel to equalized odds. In particular, we consider the axioms of resource monotonicity, consistency, and population monotonicity, all three of which relate different allocation instances to prevent paradoxes. Using a geometric characterization of equalized odds, we examine the compatibility of equalized odds with these axioms. We empirically evaluate the cost of allocation rules that satisfy both equalized odds and axioms of fair division on a dataset of FICO credit scores.


Achieving Equalized Odds by Resampling Sensitive Attributes

Neural Information Processing Systems

We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This is achieved by introducing a general discrepancy functional that rigorously quantifies violations of this criterion. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. To rigorously evaluate fitted models, we develop a formal hypothesis test to detect whether a prediction rule violates this property, the first such test in the literature. Both the model fitting and hypothesis testing leverage a resampled version of the sensitive attribute obeying equalized odds, by construction. We demonstrate the applicability and validity of the proposed framework both in regression and multi-class classification problems, reporting improved performance over state-of-the-art methods. Lastly, we show how to incorporate techniques for equitable uncertainty quantification---unbiased for each group under study---to communicate the results of the data analysis in exact terms.



On preserving non-discrimination when combining expert advice

Avrim Blum, Suriya Gunasekar, Thodoris Lykouris, Nati Srebro

Neural Information Processing Systems

Discrimination is commonly an issue in applications where decisions need to be made sequentially. The most prominent such application is online advertising where platforms need to sequentially select which ad to display in response to particular query searches.



On preserving non-discrimination when combining expert advice

Avrim Blum, Suriya Gunasekar, Thodoris Lykouris, Nati Srebro

Neural Information Processing Systems

Discrimination is commonly an issue in applications where decisions need to be made sequentially. The most prominent such application is online advertising where platforms need to sequentially select which ad to display in response to particular query searches.


Pushing the limits of fairness impossibility: Who's the fairest of them all?

Neural Information Processing Systems

The impossibility theorem of fairness is a foundational result in the algorithmic fairness literature. It states that outside of special cases, one cannot exactly and simultaneously satisfy all three common and intuitive definitions of fairness - demographic parity, equalized odds, and predictive rate parity. This result has driven most works to focus on solutions for one or two of the metrics.


A Bias Metrics A.1 Symbols For a binary classification task, we abbreviate the content in the confusion matrix as TP

Neural Information Processing Systems

Its protected label is whether the images are color or grayscale. Task (4) uses the task label "attractive" and the License: The CelebA dataset is available for non-commercial research purposes only [57]. Then a downstream classifier is trained to predict the task label on this representation. In addition to the p-value from U-test, we use Cohen's We use Levene's test to compare the magnitude of the variance between experiment The significance level is also 5%. We use an RTX 2080Ti graphics card with 11GB of memory for each training run.